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Exploring the Suitability of an English for Health Sciences Program: Model and Report of a Self-Evaluation Process 


ABSTRACT. Program evaluation is a process of carefully collecting information in order to make 
informed decisions to strengthen specific components of a given program. The type of evalua¬ 
tion an institution decides to undertake depends on the purpose as well as on the information 
the institution wants to find out about its program. Self-evaluation represents a tool that insti¬ 
tutions can rely on to follow up on the quality and pertinence of their programs. This tool can 
provide important information in relation to the relevance of outcomes, content, methodology, 
materials, and assessment along with the perceptions of the program from stakeholders such 
as students, teachers, and administrators. The results of a self-evaluation can guide the deci- 
18 sion-making process in an institution to strengthen each of the components of a set curriculum. 

This paper presents the self-evaluation process conducted by an English for Health Sciences 
program to determine the suitability of the curriculum in place. It first presents models through 
which a program can self-evaluate. Then, it elaborates on the model chosen and the steps fol¬ 
lowed in the self-evaluation process of the English for Health Sciences Program as well as the 
results obtained from the experience and the subsequent adjustments made to the program. 
The readers may use this experience as a point of reference to implement a similar process in 
their own contexts. 

Keywords: Evaluation; program self-evaluation; self-evaluation models; CBI; curriculum. 


RESUMEN. La evaluacion de programas consiste en un proceso en el cual se recoge cuidadosa- 
mente informacion con el fin de tomar decisiones informadas tendientes a fortalecer componen- 
tes especificos de un programa dado. La clase de evaluacion que una institucion decide llevar 
a cabo depende tanto del proposito como de la informacion que la institucion desee averiguar 
sobre su programa. La autoevaluacion representa una herramienta en la que pueden confiar 
las instituciones para determinar la calidad y pertinencia de sus programas. Tal herramienta 
puede suministrar informacion importante relacionada con la relevancia de los resultados, el 
contenido, la metodologia, los materiales y la evaluacion, junto con las percepciones que sobre 
el programa tengan las partes interesadas, como estudiantes, profesores y personal adminis¬ 
trative. Los resultados de la autoevaluacion pueden orientar el proceso de toma de decisiones 
en una institucion, con el fin de fortalecer cada uno de los componentes de un determinado 
curriculo. Este trabajo presenta el proceso de autoevaluacion llevado a cabo por un programa de 
ingles para ciencias de la salud, tendiente a determinar la pertinencia del curriculo en marcha. 
En primer lugar, presenta modelos por medio de los cuales se puede autoevaluar el programa. 
Luego, trabaja sobre el modelo escogido y los pasos seguidos en el proceso de autoevaluacion 
del Programa de Ingles para Ciencias de la Salud, asi como los resultados de la experiencia y los 
subsecuentes ajustes hechos al programa. Los lectores pueden utilizar esta experiencia como 
punto de referencia para implementar un proceso similar en sus propios contextos. 

Palabras clave: evaluacion; autoevaluacion de programas; modelos de auto-evaluacion; CBI; curriculo. 


RESUMO. A avaliagao de programas consiste num processo no qual se coleta cuidadosamente 
informagao com o fim de tomar decisoes informadas que visam fortalecer componentes espe¬ 
cificos de um dado programa. O tipo de avaliagao que uma instituigao decide realizar depende 
tanto do proposito quanto da informagao que a instituigao deseje averiguar sobre seu programa. 
A autoavaliagao representa uma ferramenta na qual as instituigoes podem confiar para determi¬ 
nar a qualidade e a pertinencia de seus programas. Essa ferramenta pode fornecer informagao 
importante relacionada com a relevancia dos resultados, o conteudo, a metodologia, os materiais 
e a avaliagao, junto com as percepgoes que as partes interessadas tenham sobre o programa, 
como estudantes, professores e pessoal administrativo. Os resultados da autoavaliagao podem 
orientar o processo de tomada de decisoes numa instituigao, com o fim de fortalecer cada um dos 
componentes de um determinado curriculo. Este trabalho apresenta o processo de autoavaliagao 
realizado por um programa de ingles para ciencias da saude que visa determinar a pertinencia 
do curriculo que esta sendo desenvolvido. Em primeiro lugar, apresenta modelos por meio dos 
quais o programa pode ser autoavaliado. Depois, trabalha sobre o modelo escolhido e os passos 
seguidos no processo de autoavaliagao do Programa de Ingles para Ciencias da Saude, bem como 
os resultados da experiencia e dos subsequentes ajustes feitos ao programa. Os leitores podem 
utilizar essa experiencia como ponto de referencia para implementar um processo similar em 
seus proprios contextos. 

Palavras-chave: autoavaliagao de programas; avaliagao; CBI; curriculo; modelos de autoavaliagao. 
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Introduction 


The general purpose of program evaluation is to review states or 
processes of programs and institutions (Hewitt, 1989) and provide an 
individual or a group of individuals with information to make deci¬ 
sions or judgments about such states or processes (Wigley, 1988). In 
the particular case of the educational system in the Latin American 
context, institutions have experienced a growing interest in issues 
related to the quality of education since the 1980s. This interest has 
motivated attempts to look beyond mere numbers or statistics related 
to socioeconomic factors (Fernandez, 2003). It has also motivated pro¬ 
cesses of accreditation, especially at the higher educational level, that 
have encouraged institutions to review the work they do and to allow 
for external bodies or peers to review such work as well. To review 
the quality of the education offered in an evolving context that faces 
challenges “such as increased accountability and competition, shift¬ 
ing demographics, strained financial resources, maintaining relevant 
curricula, and changing technology” (Alstete, 2006, p. 1), it is neces¬ 
sary to utilize processes of evaluation that inform stakeholders about 
strengths and areas for continuous improvement in their programs 
or institutions. As Wigley argues (in Marsden, 1991) “the purpose of 
evaluation is to improve the program and facilitate informed decision 
making (p. 34).” Processes of evaluation are key to study, inquire, ana¬ 
lyze, discuss, share, and to make conclusions about the effectiveness 
or appropriateness of educational systems in place. Along the same 
lines, Marsden (1991) points out that 

Evaluation leads to the synthesis of the data into a report 
containing a summary of the results and recommendations, with 
validated rationales, about the program being evaluated... to influ¬ 
ence decisions about the program in the future; the need for modi¬ 
fications to the program; and the need to provide cost/benefit data 
about the program, (p.36) 

This paper presents the process of self-evaluation conducted by 
an undergraduate English for Health Sciences program aiming to re¬ 
view its relevance and appropriateness by involving the perceptions of 
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target students, faculty, and program heads. The purpose of the paper 
is to share the self-evaluation experience as a point of reference for 
other institutions that may have similar interests and may find this 
information useful. 
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Literature Review 


Evaluation is considered “an applied inquiry process for collecting and 
synthesizing evidence that culminates in conclusions about the state 
of affairs, value, merit, worth, significance, or quality of a program, 
product, person, policy, proposal, or plan” (Fournier, in Mertens and 
Wilson 2012, p. 5). Evaluation is also defined by Stufflebeam and Coryn 
(2014) as “the systematic process of delineating, obtaining, reporting, 
and applying descriptive and judgmental information about some 
object’s merit, worth, probity, feasibility, safety, significance, and/or 
equity” (p. 14). Both definitions consider evaluation a process in which 
information is collected and assessed, and conclusions are made in 
relation to some set(s) of criteria. Furthermore, evaluation is a process 
that “uses inquiry and judgment methods, including: (1) Determining 
standards for judging quality and deciding whether those standards 
should be relative or absolute, (2) collecting relevant information, and 
(3) applying standards to determine value, quality, utility, effective¬ 
ness, or significance” as stated by Fitzpatrick, Sanders, and Worthen 
(2004, p. 5 )- 

Evaluation is characterized by its systematic and inquisitive na¬ 
ture, and, although it overlaps with research, it is worth highlighting 
that the concepts are not the same. While research intends to expand 
knowledge in a given field and contribute to or deepen on theories, 
evaluation helps members of groups, institutions, or communities 
who share common interests to determine the quality of the program 
or process in place. “Research seeks conclusions; evaluation leads to 
judgments” (Fitzpatrick, Sanders, and Worthen, 2004, p. 6.). 

Moreover, depending on the agents who participate in an evalua¬ 
tion, this can be referred to as external evaluation or internal / self-evalu¬ 
ation, as indicated by Stufflebeam and Coryn (2014) and Clarke (1996). 
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The first type is systematic and organized and is conducted by a body 
that is not part of what is being evaluated. The second type is “an inter¬ 
nal process of self-reflection and assessment... and should answer the 
questions: how are we doing? Are we accomplishing what we set out to 
do? And how can we improve what we are doing?” (Clarke, 1996, p. 1). 
Program evaluation, according to Spaulding, “examines programs to 
determine their worth and to make recommendations for program¬ 
matic refinement and success” (2014, p. 33). This same author points 
out that “the proximity of an evaluator to what is being evaluated 
certainly influences the access to information, the collection of that 
information, and the reporting and use of that information to promote 
change” (Spaulding 2014, p. 33). 

In her review of the historical development of internal evaluation, 
Mathison (2011) claims that in the 1960’s “university-based researchers 
were the primary source of evaluation expertise. Most program evalu¬ 
ation was contracted out” (p. 15). 

In the past, program evaluation in educational institutions con¬ 
ducted by external bodies, inspectors, or local or national assessment 
programs used to be the norm (Nevo, 2001). Nevertheless, “accredita¬ 
tion associations (in their roles of external evaluators) are now serv¬ 
ing less as decision-oriented judges and more as catalysts, advisors, 
and counselors for guiding colleges and universities in their progress” 
(Alstete 2006, p. 5). That is to say, more relevance has been given to the 
information that institutions can obtain from their own scrutiny of 
their own processes, and, therefore, educational institutions are being 
encouraged to self-evaluate rather than rely heavily on the observa¬ 
tions and feedback of external agencies. There is a current interest 
in reaching a complementary overview provided by the information 
gathered and conclusions made by external evaluators as well as the 
information and conclusions made by internal evaluators. As Nevo 
(2001) highlights, “Parallel to the almost universal phenomenon of ex¬ 
ternal evaluation, many countries have more recently tended to apply 
newly developed evaluation methods at the school level in the form 
of internal evaluation or self-evaluation” (p.96). This author further 
argues that “although internal evaluation... can also enhance account¬ 
ability, the credibility of its findings might be limited without external 
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evaluation” (p. 97). Both external and internal (or self) evaluations of¬ 
fer advantages depending on the purpose as shown in Table 1. 


Table 1 . Advantages of External and Self-evaluation 


External evaluation Self-evaluation 


“More objectivity” - the external 
evaluator is detached from the 
program under scrutiny 
Fresh outside perspective 
Credibility - especially in 
controversial environments. 
More confidence in revealing 
sensitive information. 


Better contextual knowledge 
More familiarity with stakeholders, 
which can ease anxiety 
Continuity - evaluators remain 
within organizations and can 
follow up on implementation of 
recommendations 
More practical - logistic advantages 


Thus far, a conceptualization of program external and internal (or 
self) evaluation has been given. This paper presents different models 
for program evaluation as follows. These models will be the basis for 
the self-evaluation model presented later on. 

In order for institutions to conduct internal (or self) evaluations, 
there are several models at hand that have been developed since the 
early 1900s, as presented by Guerra-Lopez (2012). One of the most 
traditional ones is the objective-based evaluation, which informs about 
student achievement of a program’s goals leading to decisions on in¬ 
structional strategies. A second model of program evaluation is the 
consumer-oriented evaluation, which focuses on “meeting consumer 
needs and societal ideals more than achieving the developer’s objec¬ 
tives for a given program” (Guerra-Lopez, 2012, p. 40). A third model 
is the discrepancy model of evaluation, which revolves around pre-estab¬ 
lished objectives and seeks to identify compliance or discrepancies be¬ 
tween what is planned and what is actually done. Another model is the 
goal-free evaluation that allows the evaluator to explore and determine 
the compliance of the program’s goals by closely studying its effects 
in all areas, regardless of the pre-established objective set by its devel¬ 
oper. A fifth model is the responsive / client-centered evaluation in which 
intentions in given processes are constantly changing and, therefore, 
there is a clear need for continuous communication between evaluator 
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and stakeholders in order to review, explore, discover, and address im¬ 
portant issues. The last model presented by Guerra-Lopez (2012) is the 
utilization-focused evaluation that concentrates on creating evaluations 
intended to inform decision makers and as Patton indicates this mod¬ 
el is implemented “for and with specified intended primary users for 
specific, intended uses” (Guerra-Lopez, 2012, p. 42). 

Rather than focusing on a single model, this paper proposes an 
eclectic or a multiple approach to program self-evaluation, combining 
elements from different models. (Bledsoe & Graham, 2015; Fitzpatrick, 
Sanders, & Worthen, 2004). 

As Fitzpatrick, Sanders, and Worthen (2004) point out, when eval¬ 
uators adopt multiple models, they “can ensure a better fit by snipping 
and sewing together bits and pieces of the more traditional ready¬ 
made approaches and even weaving a bit of homespun, if necessary, 
rather than by pulling any existing approach off the shelf. Tailoring 
works” (p. 164). Along these lines, the self-evaluation model proposed 
in this paper explored the achievement of program objectives by an¬ 
alyzing student achievement at different levels of expected language 
proficiency. It also focused on the perceptions of stakeholders such as 
students, heads of department, and teachers, considering that their 
motivations and the contextual demands are in constant evolution. 
Additionally, the model reviewed the alignment existing within the 
different elements of the curriculum with the purpose of identifying 
possible discrepancies in such alignment. In this sense, the self-eval¬ 
uation aimed to obtain an overall view of the program by using fea¬ 
tures from different models. This combination allowed for access to 
information resulting from the application of different data collection 
instruments to students, teachers, and program administrators. This 
information was complemented and/or contrasted with document 
analysis (curriculum, textbooks) allowing for a look at the program 
from different perspectives. This self-evaluation process is in line with 
the one suggested by the Colombian National Council of Accredita¬ 
tion (cna, 2013), which indicates, “the success and seriousness of a self 
evaluation process requires that the institution takes leadership of the 
process and encourages significant participation of the academic com¬ 
munity.” (p. 7, own translation). 
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Context of the self-evaluation 
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English for Health Sciences (ehs) is a program designed for undergrad¬ 
uate students of Medicine and Dentistry at a private university on the 
Caribbean Colombian Coast. It consists of five 8o-hour courses and 
an-8o-hour preparatory course that add up to 480 hours of instruction. 
The average number of students enrolled in the program per academic 
year is around 800. 

Students enrolled in the Health Sciences Program come from 
different socioeconomic backgrounds. Some of them come from ru¬ 
ral areas from the Department of Atlantico and nearby departments 
on the Colombian Caribbean coast. The rest of the population is from 
Barranquilla, the city in which the university is located. The School of 
Health Sciences has a very demanding admissions process in place, 
which is why, in general, students who enter the programs offered by 
this school have outstanding academic records of performance. This is 
true about general areas of knowledge, but not necessarily for English. 
Around 50% of these students are placed in the Ai or A2 levels of lan¬ 
guage proficiency according to the Common European Framework of 
Reference (cefr), and around 25% of them are exempted from taking 
English courses. These results are determined by the placement exam 
that incoming students take at the beginning of their studies in the 
university. Based on the particular characteristics of those students 
enrolled in the ehs Program and the amount of exposure to class in¬ 
struction, the main goal of the program is to help students reach a Bi 
level of language proficiency. 

The English for Health Sciences Program aims to develop stu¬ 
dents’ communicative skills, so they can interact in general situations 
of everyday life and in specific contexts related to the area of health 
in which the use of English is required. In order to do so, it uses two 
different approaches: levels one through three focus on the develop¬ 
ment of language skills through an integrated approach to language 
teaching. The upper levels, levels four and five, use content-based 
instruction and English for Specific Purposes, respectively, and they 
aim to strengthen language skills through the use of discipline-related 
content. The program is made up of around 15 language teachers, who 
are coordinated by two experienced fellow colleagues, one of them an 
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experienced English language teacher and the other an English teach¬ 
er with a bachelor’s degree in medicine. 

The next section outlines the steps followed to carry out the self- 
evaluation of the ehs program. It starts with the description of the 
process of planning the self-evaluation; subsequently, it discusses 
the actions taken to implement the evaluation, and finally, the report¬ 
ing phase of the self- evaluation process is presented. 

The self-evaluation model 

As previously stated, an eclectic or a multiple approach to program 
self-evaluation model was adopted so that it would allow evaluators to 
take some elements from different approaches to broaden the picture 
and obtain information from different perspectives. The evaluation 
process was mainly focused on three stages: planning, conducting, 
and reporting the findings on the self-evaluation. During the planning 
phase, evaluators had to make decisions regarding the purpose, de¬ 
sign, questions, resources, and procedures. Once decisions were made, 
the implementation phase started. In this phase, evaluators gathered 
information from all the sources available to analyze it. Finally, the 
last phase reported the findings, and generated recommendations for 
improvement. Figure i shows the steps followed in the process. 

The Planning Phase 

The starting point of any self-evaluation process requires reflection 
on the purpose and the procedures that are going to be followed as well 
as the preparation of resources that will be needed during the various 
stages of the evaluation. As it was previously mentioned, some deci¬ 
sions related to purpose, participants, resources, times, and instru¬ 
ments to conduct the experience had to be made when planning the 
self- evaluation. An important initial decision was to define the leaders 
of the self- evaluation process and their responsibilities. Therefore, a 
leading committee, the Self-Evaluation Committee (sec), was created. 
The sec was made up of a team of three faculty members: two experi¬ 
enced teachers in the ehs program, one of them an experienced teach¬ 
er and the other a teacher-physician, and the academic coordinator of 
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the program. They had the responsibility of planning, conducting, and 
reporting on the self-evaluation. Additionally, they were supported 
by other fellow teachers in the program who helped them formulate 
the questions for the data collection instruments and analyze some 
of the data gathered, among other things. 
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Figure 1 . The self-evaluation model. 
Adapted from “Program Evaluation. Alternative Approaches and 
Practical Guidelines” by Fitzpatrick, Sanders, and Worthen (2004). 
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At the beginning of the planning phase, the sec identified differ¬ 
ent evaluation models in the field as well as effective evaluation strat¬ 
egies and curricular evaluation frameworks used by institutions and 
programs in similar contexts. Through this literature review process, 
the sec team was able to define and design a suitable evaluation model 
to follow as well as the questions that would guide the process. 

Taking into account that the main purpose of conducting a 
self-evaluation in the ehs program was to find out whether the pro¬ 
gram was relevant and appropriate for all the stakeholders, the follow¬ 
ing guiding questions were defined: 

» How relevant is the ehs program to the current needs of Health 
Sciences students? 

» Which language needs do students in the Health Sciences Pro¬ 
gram have? 

» How effective is the English for Health Sciences Program in terms 
of the fulfillment of objectives and students development of En¬ 
glish proficiency? 

These questions were aimed to provide the necessary information 
regarding the relevance and appropriateness of the program in place. 
After evaluators formulated the guiding questions, it was necessary 
to determine available data sources and resources in order to be able to 
answer those questions appropriately. New questions (such as “what 
information do we need?”; “who knows this information?”; “how, and 
who can gather that information?”) needed to be answered before the 
next step was taken. Those answers gave the sec useful insights relat¬ 
ed to evaluators, participants, instruments, and procedures. 

The final task in planning a self-evaluation study is to describe 
how it will be carried out (Fitzpatrick, Sanders, and Worthen, 2004) 
since there are many tasks and resources to take into account during 
a specific time frame; “a management plan is essential to help in over¬ 
seeing the project” (p. 275) and to constantly monitor the achievement 
of tasks and the availability of resources in order to conduct a thorough 
and systematic program evaluation. In the case of the self-evaluation 
presented in this paper, this management plan was given the title: 
action plan. The action plan form, shown in appendix 1, included the 
stages and tasks planned as well as the time frame allocated to each of 
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the activities. The action plan helped organize the process, as well as 
keep track of task progress. Once the action plan was confirmed, the 
implementation stage started. 

The Implementation Phase 

For the purpose of this self-evaluation process, the sec was in charge 
of the collection and analysis of data. The information gathered was 
discussed, reviewed, and analyzed in periodic meetings for the pur¬ 
pose of standardizing the process, drawing conclusions, and proposing 
an action plan for improvement. 

There were two main sources of information in the self-evaluation: 
the institutional documents and the program stakeholders. Therefore, the 
methods for data collection chosen were document analysis, focus groups 
(for both teachers and students of the ehs program), surveys to students 
only, and interviews with the Heath Sciences department heads. 

The first method, document analysis, supplements data collected 
from other qualitative sources because of its practicality, availabili¬ 
ty, cost-effectiveness, stability, and exactness (Bowen, 2009). In this 
particular case, the sec analyzed the syllabi of the program, students’ 
grades, and the exit proficiency level reports from five consecutive 
academic terms (2013, 2014 and first semester 2015). The English for 
Health Sciences program was analyzed to determine the suitability 
of its goals as well as the alignment of the learning outcomes with 
those goals, and the appropriateness of the instructional materials, 
methodology, and assessments. The analysis of students’ grades was 
carried out for the purpose of quantitatively identifying whether the 
goals of the program had been met or not in terms of pass-fail rate, 
since passing the course would mean that the corresponding learn¬ 
ing goals for the level were achieved. Whereas the pass-fail rate was 
not the only evidence of students’ achievement, it provided a general 
view of students’ achievement in each course. Likewise, the analysis of 
students’ exit level proficiency reports showed complementary infor¬ 
mation on their achievement once they completed the whole program. 
For example, the pass-fail rate in 2013 and 2014 (see table 2 and table 3) 
showed that on average more than 90% of students in the program 
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met the requirements to pass their English courses, with the exception 
of second semester in 2014, which showed a lower percentage 87%. 
This can be interpreted as a successful process since there are low fail 
rates, which may indicate that most students reached the expected 
goal for the program. 

Table 2. Pass-fail Rate EHS 2013 


EHS 2013 



I s ' 

Semester 




2 Nd Semester 



Level of the 
Program 

Students 

Enrolled 

Pass Rate 

Fail Rate 

Students 

Enrolled 

Pass Rate 

Fail Rate 

Nivelatorio 

19 

16 

84% 

3 

16% 

3 

3 

100% 

- 

0% 

1 

45 

43 

96% 

2 

4% 

25 

24 

96% 

1 

4% 

2 

57 

55 

96% 

2 

4% 

41 

97 

90% 

4 

10% 

3 

56 

56 

100% 

- 

0% 

66 

63 

95% 

3 

5% 

4 

74 

65 

88% 

9 

12% 

57 

51 

89% 

6 

11% 

5 

64 

54 

84% 

10 

16% 

60 

56 

93% 

4 

7% 

Total 

315 

289 

92% 

26 

8% 

252 

233 

93% 

19 

7% 


Table 3. Pass-fail Rate EHS 2014 


EHS 2014 



1 st Semester 




2 Nd Semester 



Level of the 
Program 

Students 

Enrolled 

Pass Rate 

Fail Rate 

Students 

Enrolled 

Pass Rate 

Fail Rate 

Nivelatorio 

19 

17 

89% 

2 

11% 

6 

6 

100% 

0 

0% 

1 

32 

31 

97% 

1 

3% 

26 

23 

88% 

3 

12% 

2 

44 

44 

100% 

0 

0% 

31 

26 

84% 

5 

16% 

3 

48 

47 

98% 

1 

2% 

38 

27 

71% 

11 

29% 

4 

48 

44 

92% 

4 

8% 

45 

44 

98% 

1 

2% 

5 

50 

42 

84% 

8 

16% 

32 

28 

88% 

4 

12% 

Total 

241 

225 

93% 

16 

7% 

178 

154 

87% 

24 

13% 
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However, student achievement requires more careful analysis 
when compared with students’ exit level through the results of an in¬ 
ternational standardized test that all of them take as a way to monitor 
their language proficiency once they have completed the program. In 
the same years, around 75% of students obtained the expected result 
Bi in this standardized test. Nevertheless, the remaining percentage 
scored under the expected level. There are important variables that 
may influence the results students obtain in this standardized exit 
test such as students’ minimum effort or lack of test taking abilities, 
among other. However, assessments in place as well as the minimum 
set score for students to pass the different levels in the program are 
worth continuous review. This review may serve to explore a better 
alignment among current outcomes, instruction, and assessments. 

The other qualitative data collection instruments selected: focus 
groups, surveys, and interviews, aimed to identify stakeholders’ per¬ 
ceptions and expectations of the program. Focus groups have become 
increasingly popular in the educational field. They are highly recom¬ 
mended in program evaluations since they provide the opportunity to 
have individuals elaborate on their thoughts or opinions about partic¬ 
ular issues of a program. Rennekamp and Nall (2008) argue that focus 
groups are very useful in program development and evaluation since 
they can be used to determine program needs, design new programs, 
improve a program in place, identify customer satisfaction, make and 
test policies, and evaluate outcomes. In our particular case, focus 
groups were used to identify students, teachers, and alumni percep¬ 
tions about the suitability of the English for Health Sciences program 
in place. As an example, the questions asked in the teachers’ focus 
group are shown in appendix 2. 

To ensure objectivity in the collection of information, the focus 
groups were conducted by an external professional, a psychologist, 
who was not part of the program. Participants were randomly chosen, 
and questions were carefully designed and reviewed to make sure they 
reflected participants’ perceptions in terms of learning, methodology, 
materials, content, and assessments. Focus groups were conducted at 
a time and place scheduled, and the external professional used the 
questions designed; however, she would add further questions as 
needed in order to deepen a point of discussion. After the meeting 
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with students, the external professional analyzed and contrasted all 
the comments and wrote a report with the main findings resulting 
from the focus group. Table 4 illustrates the main points taken from 
the report given by the external professional: 

Table 4. Students’ Focus Group Main Points 


EHS Students’ focus group 2015 

In terms of students’ perception of their own learning process, students in 
upper levels considered that there was actual progress in their language 
learning process; however, students in the lower levels thought that the levels 
were too basic and, rather, seemed to be a review of what they had learned 
in high school. Something they highlighted though as a strength in levels one 
through three was their oral skills reinforcement, which most of them did not 
have the opportunity to develop at high school as expected. 

In terms of methodology, students expressed their satisfaction with their 
teachers’ teaching styles. They acknowledged the benefits of the empathetic 
teacher-student relationship in the classroom and the positive collaboration in 
the learning context. 

In relation to content, the discipline-related component in level four appears to 
encourage higher levels of motivation in students whereas students in levels 
one through three find the content repetitive and basic. 

Regarding particular language skills development, students in general ar¬ 
gued that all skills are important; however, they agreed on a higher need for 
reading and listening development considering what they are required to do 
as health sciences students. 


While focus groups allow for the participants to elaborate on their 
opinions and perceptions, surveys allow for coverage since they can 
be applied to larger populations. In this sense, these instruments are 
complementary in terms of depth of information and the quantity of 
the sample. Taylor-Powell and Hermann (2000) recommend the use 
of surveys when one needs a report of things that are not observable 
such as people’s opinions and beliefs. This information should be pro¬ 
vided by the participants themselves. In the self-evaluation presented 
in this paper, evaluators used an online survey due to its practicality in 
terms of collecting and analyzing the information gathered. 136 cur¬ 
rent students took the survey and provided information related to 
their language learning motivations, needs, and wants (Appendix 3). 
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Finally, a semi-structured interview with the Health Sciences Aca¬ 
demic Director and the Program Coordinator was conducted in order to 
identify what they expected from the English for Health Sciences program 
and what they perceived, as students’ main language needs. According to 
Kvale (1996), interviews are conversations aimed to understand the world 
from the subjects’ point of view. The main challenge for the program 
evaluator is “to provide a framework within which people can respond in 
a way that represents accurately and thoroughly their point of view about 
the program” (Patton, 1987). In that sense, the sec carefully designed a 
set of six questions to guide the interview with the Health Sciences de¬ 
partment heads (Appendix 4). However, the interviewer was free to vary 
the wording and the order of the questions according to the respondent’s 
answers in order to allow for discussion of any important topic raised 
during the interview. The interview was recorded and then analyzed by 
the sec. Some of the main points made by ehs Academic Director and the 
Program Coordinator are indicated in table 5. 

Table 5. Interview with Health Sciences Academic 
Director and Program Coordinator Main Points 


Interview HS Academic Coordinator and Program Coordinator 2015 

For about the last four three years, communication between the Health Scienc¬ 
es School and the English for Health Sciences program has improved, which 
is reflected in a better flow of administrative processes and attention to partic¬ 
ular students’ cases when needed; e.g., the Head of Department said, “I think 
things have improved. Administrative issues work better because we have im¬ 
proved our channels of communication between programs.” (own translation) 

The Heads of the English for Health Sciences School as well as students think 
that the language skills students need the most are reading and listening, fol¬ 
lowed by speaking. They do not consider that writing development is essential 
at this point of students’ academic life; e.g., the Head of Department high¬ 
lighted that students “need all the language skills, however, reading is very 
important. In the area of medicine, you have to read a lot." (own translation) 

They have learned from students that upper levels are more academically de¬ 
manding; however, they argue that students seem to enjoy upper levels more 
than they do lower ones; e.g., the Program Coordinator said, “we hear from 
students that levels IV and V more difficult and consequently, they fail more; 
however, there is no problem with the content. Students’ comments revolve 
around the level of difficulty, (own translation) 
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Once all the information was gathered, the sec held regular meet¬ 
ings in order to analyze and validate the data from the different sourc¬ 
es through the process of triangulation. After the cross verification of 
information, the sec was able to answer the self-evaluation questions 
with the information provided by different stakeholders through a va¬ 
riety of sources. Subsequently, the sec was able to report the findings 
and structure an action plan for program improvement based on these 
findings. In the next section, the last stage of the self- evaluation pro¬ 
cess, the reporting phase, will be described. 

The Reporting Phase 

The findings of the self-evaluation process should ideally be shared 
with program stakeholders in order to promote and facilitate the deci¬ 
sion-making process for program improvement. For this to happen, it 
is highly recommended to write a final evaluation report. The National 
Center for Chronic Disease Prevention and Health Promotion (2013) de¬ 
fines a final evaluation report as “a written document that describes 
how you monitored and evaluated your program. The written report 
allows the evaluators to describe the 'What’, the ‘How’, and the ‘Why it 
matters.’” In terms of the “What,” the report should clearly state what 
the purpose of the self-evaluation was, what the findings were, and 
what those findings mean for the program. The “How” should be de¬ 
scribed as how the self-evaluation was conducted (times, activities, 
instruments, analysis, etc.). Finally, the “Why it matters” should reflect 
the importance of the findings for the program as well as the adjust¬ 
ments that could be made after the self-evaluation. 

The process of writing the report may seem a bit overwhelming 
due to the amount of information gathered in the previous phase. 
Therefore, it is recommended to always have the evaluation questions 
in mind in order to keep the report meaningful and relevant. Fur¬ 
thermore, it is also advisable to present the findings in an organized 
manner, so that it is easy for the audience to draw conclusions and to 
create a plan of action for improvement. In our case, the sec decided 
to organize the information related to every aspect of the program that 
was evaluated in terms of strengths and areas of improvement. This 
way, it was practical to show what features of the English for Health 
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Sciences program were relevant and effective, what needed minimal 
adjustment, and what required a complete change. A sample of the 
reporting charts used in the final report section of the program can be 
found in Appendix 5. 

Based on what has been said about the evaluation report, the 
most relevant information presented is related to the self-evaluation 
findings, conclusions and recommendations. Those recommendations 
or actions for improvement resulted from the self-evaluation process 
would be the final stage of a process and the starting stage of a new 
one since program evaluation an ongoing, cyclical process. 

In brief, these were the answers to the self-evaluation questions: 

» How relevant is the ehs program to the current needs of Health 
Sciences students? 

- Both students and academic representatives of the Health Sci¬ 
ences School consider that the upper levels (four and five) of the 
English for Health Sciences program fulfill their expectations in 
terms of the content developed through the study of language. 
Based on students comments in the focus groups (table 4), they 
seem to be motivated by the content they study in upper levels; 
however, they do not seem aware of the importance of recy¬ 
cling content in language learning since they find the content 
in levels one though three repetitive; they also find the latter 
somewhat basic according to their expectations. 

» Which language needs do students in the Health Sciences Pro¬ 
gram have? 

- Both students and academic representatives of the Health Sci¬ 
ences School agreed on the skills that a medical student needs 
at the undergraduate level. They consistently referred to reading 
and oral skills. The first type of skills to consult bibliographies 
produced in English, and the second to be able to understand 
spoken texts dealing with topics of their professional interest. 

» How effective is the English for Health Sciences Program in terms 
of the fulfillment of objectives and students development of En¬ 
glish proficiency? 
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- There seems to be a need for a better alignment among outcomes, 
instruction, and assessments. Student achievement throughout 
the different levels of the program shows very positive results 
with low fail rates. However, on completion of the program, 
over 25% of the student population obtained lower results than 
expected, according to the standardized English exam applied 
in level five. This requires a curriculum review, especially the 
established goal in terms of the language proficiency level for 
students to reach as well as a review of the exam that is being 
applied and its coherence with the English program. 

Appendix 6 shows some of the actions proposed as an action plan 
for program improvement as a result of the self-evaluation. The next 
step is following up on the implementation of those actions and that 
would be the starting point of a new program evaluation process. 


Discussion 


This paper was aimed to present a self-evaluation process carried out 
in an English for Health Sciences Program, in the hopes that it can 
serve as a source of reference for institutions that may have interests in 
self-evaluating their programs. This self-evaluation was found to be an 
enriching experience for both the faculty and participants involved for 
several reasons. First, faculty were actively engaged in exploring models 
of self-evaluation and in defining which of those models could be best 
applied in our context. The defining and selection of a model was fol¬ 
lowed by active teamwork in establishing the areas to be evaluated and 
the tools to be used to gather information for the program analysis 
and evaluation. All of these actions allowed for the joint-construction 
of a structured framework for the evaluation. Second, the program 
as a whole benefited from its stakeholders’ feedback. Academic pro¬ 
gram heads, teachers, and students were given a space to voice their 
perceptions of the program for the purpose of improvement. In this 
way, the English for Health Sciences program was analyzed from var¬ 
ious perspectives. Third, the self-evaluation fostered a space for and 
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encouraged a critical view of the internal work and practices of the 
program. It was found that adopting a critical approach in the self-eval¬ 
uation towards all components of the program is difficult at first, but, 
once it is adopted, it can be very enriching for everyone involved. 

From the self-evaluation presented, there are also some challeng¬ 
es worth mentioning for programs to bear in mind when going through 
similar processes. The first challenge is the allocation of both human 
and material resources necessary in order to conduct the self-evalua¬ 
tion. Programs need to determine the people who will be leading the 
process and the conditions under which these processes will be car¬ 
ried out. This may be a challenge when faculty have to take on extra 
responsibilities related to the self-evaluation while also fulfilling their 
usual work duties. In terms of material resources, there are expenses 
that need to be considered including: external professionals to conduct 
focus groups, photocopies or technological resources for mass surveys, 
and assistants to help with logistics, among others. According to our 
experience, planning is crucial to facilitate good management of both 
kinds of resources. Another challenge to be aware of is the selection of 
the self-evaluation team. It is necessary to count on individuals who 
are familiar with the different processes within the program, who are 
empathetic towards other team members, and who are willing to con¬ 
tribute to the process through a critical point of view. Team members 
spend significant time planning, designing, analyzing, and reporting. 
Therefore, a cooperative environment is key to success. A third chal¬ 
lenge is related to access to information and target participants during 
the self-evaluation process. Making appointments and counting on the 
availability of program heads and student alumni, for example, can 
be troublesome. Consequently, perseverance is of great importance 
in keeping with the evaluation timeline and in gathering information 
from the target sources. 

Overall, program self-evaluations have both benefits and chal¬ 
lenges for all parties involved in the process. Whatever the context, 
conducting a program self-evaluation, based on the experience pre¬ 
sented, is extremely valuable as it allows for a stronger awareness and 
better understanding of program processes and practices. 
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Conclusion 


The experience of self-evaluating the English for Health Sciences 
Program presented in this paper offered its faculty an opportunity to 
develop stronger awareness of the strengths and areas for improve¬ 
ment of the program and a better knowledge of it as a whole. For in¬ 
stance, the open and continuous communication between the English 
Program and the School of Health Sciences must be maintained and 
reinforced. Throughout recent years, this is an area that has improved 
the administrative organization of both programs; however, this com¬ 
munication needs to further improve and establish stronger academic 
connections between English and the discipline related subjects that 
students take in their program of study. The self-evaluation provided 
information related to program goal, objectives, methodology, content, 
student achievement, and student and program needs that represents 
valuable input to propose program action plans. The development of 
this knowledge and awareness and the adoption of self-evaluation as 
a systematic process can help the program direct its efforts to contin¬ 
uously improve. 

Continuous evaluation must be an inherent element to educational 
programs in order to guarantee the quality of the education offered to 
students. The more carefully institutions look at their processes, the 
better prepared they can be to identify and improve their weaknesses. 
Evaluations should not always be imposed and conducted by outside 
experts. They should be conducted systematically by members of the 
institution, who can then use the concept of an external peer to com¬ 
plement or reinforce their own knowledge and views of their program. 
Self-evaluation is, therefore, a strategy that institutions should adopt 
in the pursuit of continuous improvement, self-awareness, and growth. 
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Action Plan for EHS SELF EVALUATION 


Activity, Output 
and Sub-Activities 


FEB 

MAR 

APR 

MAY 

JUN 

JUL 

AUG 

SEP 

OCT 

NOV 


DEC 


1. Planning 


1.1. Review of literature on the English language field (curriculum evaluation, CLIL, 
CEFR) 


Identify successful 
models in the field 

as well as effective 

pedagogical 

strategies 

X 











Define frameworks 
to curricular review/ 

evaluation 

X 












1.2. Developing a self-study Action Plan 


Draft self-study 

Action Plan and 
present it to 
stakeholders 


X 










Finalize self-study 
Action Plan 


X 










Revise self-study 
Action Plan as 
required 


X 










Present Self-study 
objectives and 
action plan to the 
Director of Instituto 

de Idiomas 


X 











2. Implementation 


2.1. Document Analysis- Reviewing Curriculum 

Identify needs 
(language and 
content) established 
by the HS programs 
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Action Plan for EHS SELF EVALUATION 


Activity, Output 
and Sub-Activities 

FEB 

MAR 

APR 

MAY 

JUN 

JUL 

AUG 

SEP 

OCT 

NOV 

DEC 

Evaluate the 

relevance of 
learning outcomes 
in the EHS 


X 

X 









Establish the 
alignment among 
the SLOs, 
assessment, 
methodology, and 
materials 


X 

X 









Analyze statistics 
related to Alumni 
determined by their 
English proficiency 





X 







Analyze students' 
progress on the 
basis of placement, 
pass-fail rate, 

TOEFL results 





X 







Identify students’ 
perceptions of the 
program in general 
through course 
evaluation analysis 




X 








Determine teachers’ 
profile in the EHS 
program to identify 
strengths and areas 
for improvement 




X 








Report on progress 




X 









2.2. Data collection from stakeholders 


2.2.1. Conducting focus groups 

Identify target 
needs, learning 
needs, perception 
of the program, 
and expectations of 
current students 
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Action Plan for EHS SELF EVALUATION 


Activity, Output 
and Sub-Activities 

FEB 

MAR 

APR 

MAY 

JUN 

JUL 

AUG 

SEP 

OCT 

NOV 

DEC 

Identify teachers’ 
perceptions on 

SLOs, evaluation, 
materials, 
methodology, 
and students’ 

achievement 





X 








2.2.2. Applying surveys 


Get/obtain feedback 
from alumni in terms 

of the relevance 

and usefulness of 
the EHS for their 
practice 






X 






Identify target 
needs, learning 
needs, perception 
of the program, 
and expectations of 
current students 




X 









2.2.3. Making interviews 


Determine 
opportunities 
offered by the 
university to 

HS students in 
which English is a 
requirement 





X 







Identify the 

HS program 
expectations in 
terms of students’ 
English proficiency 
and use. 





X 







Report on progress 






X 







3. Reporting 


3.1. Writing the document 

Define sections of 
the self-study 
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Action Plan for EHS SELF EVALUATION 


Activity, Output 
and Sub-Activities 

FEB 

MAR 

APR 

MAY 

JUN 

JUL 

AUG 

SEP 

OCT 

NOV 

DEC 

Consolidate 

information 








X 

X 

X 


Finalize the 

document 










X 

X 


3.2. Presenting results 


Arrange and 
conduct meeting 
with Curricular 

Committee and 
the Director of II to 
present findings 











X 

Design an action 
plan to make 
the necessary 
adjustments to the 
program 











X 
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English for Health Sciences Program Teachers Focus Group 

Aim: To get feedback from / know the opinion of teachers about the pro¬ 
gram in order to gather information for the program self-evaluation. 

I. Learning and Content: 

1. Do you feel that your students demonstrate progress in their language 
learning as they advance through the different levels of the program? 
Explain. 

2. Do you think that the approaches used in the program (general English 
for levels 1-2-3 and teaching English through Content for Level IV-V) 
allow students to learn English effectively? Explain. 

3. If you could change anything from the English for Health Sciences Pro¬ 
gram (content, textbook, methodology or evaluation), what would you 
suggest to change? 

4. In terms of skills (speaking, listening, reading, writing), which do you 
think may be most useful to your medicine students (and now dentistry) 
in their professional and academic life? Why? 

5. What do you think about the learning outcomes of the different levels? 
Are they clearly stated in the syllabus? Do the learning outcomes allow 
you to plan and guide your work at the level? 

II. Assessment: 

6. Do you think that the written tests (midterm, final, unit achievement test) 
account for the real English writing competence of your students? Ex¬ 
plain. 

7. Do you think that oral assessments (midterm, final, unit achievement 
test) account for the real language speaking proficiency of your stu¬ 
dents? Explain. 

8. According to your experience teaching courses in the program, what 
is your opinion about the assessed writing processes (writing assign¬ 
ments)? 

III. Materials, Methodology: 

9. Do you think that the texts used in classes support the teaching-learn¬ 
ing process? Explain your answer. 

10. What extra materials (other than the textbook) do you think facilitate the 
teaching-learning process in the classroom? Why? 

IV. General Aspects: 

11. What strengths and areas of improvement do you currently identify in 
the English for Health Sciences Program? 
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Appendix 3 


English for Health Sciences Program Current Students Survey 

Answer the following questions according to what you have experienced 
in the program. Your answers are valuable to improve our programs and to 
offer academic processes that fit your needs. 

LEARNERS’ NEEDS 

1. In your particular case, what is the main reason you are learning English? 

□ to do well at school 

□ to interact in English-speaking countries (or abroad) 

□ to access job opportunities 

□ to be able to get more opportunities to study 

□ to meet an academic requirement 

□ Other. Explain. 

2. What emphasis would you like to have for the English Program you are 
currently learning? 

□ English for everyday use (day to day use) 

□ English for health sciences 

□ No preferences 

3. Which of the following skills do you think is the most important for you 
to develop in English? (Choose one option) 

□ reading 

□ writing 

□ speaking 

□ listening 

4. What is your strongest ability? (Choose one option) 

□ reading 

□ writing 

□ speaking 

□ listening 

5. What is your weakest ability? (Choose one option) 

□ reading 

□ writing 

□ speaking 

□ listening 
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6. What skill do you need most often in the academic activities of your 
field? 

□ reading 

□ writing 

□ speaking 

□ listening 

7. What topics or content do you think you need to learn in English? 


LEARNERS’ WANTS 

8. What do you like to do most in English? (Rank the following skills in 
order of preference: 4 you like the most -1 you like the least) 

□ reading 

□ writing 

□ speaking 

□ listening 

9. How do you like to learn English in class? (Choose three options) 

□ studying grammar rules 

□ working on writing exercises 

□ writing short texts 

□ reading texts/stories 

□ listening to audio 

□ watching videos 

□ working on oral exercises 

□ participating in group discussions 

□ working on projects 

□ reading out loud 

□ working with computers 

□ working on the Internet 

□ playing 

□ other: explain_ 

10. How do you prefer to work in your English classes? (Choose one option) 

□ alone 

□ in pairs 

□ in groups 

□ as a whole class 
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11. What material do you like to use in your English class? (Rank the follow¬ 
ing material in order of preference: 5 you like the most - 1 you like the 
least) 

□ Textbook 

□ In-house material designed to meet the specific needs of the program 

□ Online material (written and oral) 

□ Realia (real-life objects taken to the classroom to teach) 

□ Software 


12. What general topic or content would you like to learn in English? 



Appendix 4 


English for Health Sciences Program Department Heads Interview 

Questions 

1. What do you expect students to be able to do in English on completion 
of the English program? 

2. What English language goals (if applicable) does the School of Health 
Sciences have for undergraduate students? 

3. What English language requirements does the health sciences pro¬ 
gram have in its curriculum? 

4. What academic activities does a Health Sciences student do in which 
he/she needs English proficiency? At what level? 

5. What subjects of the Health Sciences programs have explicitly estab¬ 
lished the need for a language component? 

6. What opportunities for language use do Health Sciences undergradu¬ 
ates have? 
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Outcomes Final Report Sample 


Strengths 

Room For Improvement 

» Alignment with the 

» Unclear / confusing target outcomes in some 

CEF in levels 1-3 

levels. 

» Outcomes in all 

» No clear evidence of skills progression 

skills throughout 

throughout the program. 

the program. 

» No specific description of text / task progression 

» Discipline-related 

in reading skills. 

outcomes in levels 

» Alignment with CEFR in levels 4 & 5 needs 

4 and 5. 

revision. 


» Unclear distinction between language and 


content learning outcomes in levels 4 & 5 for 


listening and reading 


» Dentistry program outcomes are not mentioned. 


Appendix 6 


ACTION PLAN 2016 

1. Revise the progression, content, and alignment with the cefr of the dif¬ 
ferent learning outcomes throughout the program 

2. Describe the different tasks and texts students will be exposed to in the 
different levels. 

3. Evaluate the textbooks that are currently used for instruction as well as 
possibly piloting materials for the first semester 2016. 

4. Assess the relevance of writing instruction and assessment throughout 
the program. 

5. Include explicit learning strategy instruction in the program. 

6. Offer more emphasis on health-related vocabulary as well as reading 
material in the first three levels. 

7. Make sure the assessment tasks and instruments are aligned with the 
content and instruction. 

8. Work on a health related reading program as supplementary material 
for the courses. 

9. Revise and design new assessment tools for the levels. 

10. Look for formative assessment options in the program. 
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